Enhancement of video data using motion-compensated postprocessing techniques

2. Improvement of the spatial resolution of the sequence, interpolating gray (or color) values at fractional spatial positions both in the known (transmitted) and the unknown (interpolated) frames. In many video coding schemes, especially at low bitrates, spatial and temporal subsampling of the image sequences is considered. This is realized by leaving out rows and columns from the images, and skipping whole frames at the transmitter. To get the best possible quality image sequence at the receiver side, the skipped portion of the video should be reconstructed using advanced motion-compensated (MC) postprocessing techniques. 3. Removal of artifacts caused by subsampling, loss of data, distortion, etc. Our paper mainly addresses the first problem. Enhancement of the temporal resolution of an image sequence requires interpolation of image frames at time instances, where information was not transmitted, or has not been recorded. Good quality interpolation can only be done using advanced MC frame interpolation algorithms. In this paper an MC frame interpolation algorithm is presented, based on a pel-recursive motion estimation algorithm [1], and utilizing an accelerated motion model [4], motion-based segmentation [5] with proper handling of covered and uncovered areas [6]. Our paper mainly focuses on the restoration / generation of unknown frames of the sequence at time instances, where the original scene has not been sampled, or which were skipped from the original sequence in the transmitter. This enhancement of the temporal resolution is performed using our advanced MC interpolation algorithm, utilizing an accelerated motion model and motion-based segmentation with proper handling of covered and uncovered areas. The algorithm can be used to avoid jerkiness and blurring of the restored image sequences. The basic techniques used in our algorithm are described in the following sections.